REMARKS 

Reconsideration and allowance are respectfully requested in view of the foregoing 
amendment and the following remarks. 

This amendment changes the numbering of claims 60-69. These claims were 
entered in a preliminary amendment and their numbering was erroneous. The claims as 
presented in this amendment have new numbers 51-60. The office action appears to have 
treated the original, misnumbered claims as though they were numbered 51-60. 

Claims 1 1-22 and 27-60 were rejected under 35 U.S.C.103(a) as being 
unpatentable over Maurer (US 6,272,231) in view of Singh (US 6,204,860). Claims 23- 
26 were rejected under 35 U.S.C.103(a) based upon the Maurer and Singh references and 
further in view of Ferre (US 6,175,756). 

The following remarks will show that the office action is based upon erroneous 
findings and that the claims are patentable over the art of record. 

The Maurer reference fails to show at least three steps out of the six steps of claim 
1 1 . The Maurer does not mark salient features of an object, does not acquire a plurality 
of an initial 2-D images of the object, and it does not calculate 3-D locations of the salient 
features of an object. 

Maurer does not show or suggest marking salient features of an object with 
markers for motion tracking. The office action relies upon column 5 lines, 29-46 of 
Maurer. However, no where that portion of Maurer does the patent show or suggest 
marking an object with markers. In column 5 lines 29-46 the art discusses marking node 
locations of a bunch grab 70. However, the bunch grab 70 is not an object nor is it even 
an image of an object. Instead, the bunch grab 70 is merely a collection of lines that are 
connected together. A collection of lines on a piece of paper is not the same as a three 
dimensional object with physical markers. Likewise, the nodes referred to in that section 
or even the jets 72 are not markers. Instead, the nodes are no more than intersections of 
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lines in a graph. Nothing in Maurer describes, shows or suggests any physical markers 
that are connected to a physical object. Instead, Maurer is only concerned with an 
abstract graph that has intersecting points (nodes). 

The Maurer does not calculate 3-D locations of the salient features of the object. 
The rejection relies on column 10, lines 51-67 in Maurer. However, that section in 
Maurer simply refers to a three dimensional finding technique that is similar to the earlier 
discussed two-dimensional technique. No where does Maurer show or suggest that the 
three dimensional landmark finding technique is or provides 3-D locations of the object 
in accordance with the calibration parameter of the camera. Indeed, it appears that the 
camera calibration is entirely irrelevant to either the 2-D technique or the 3-D technique. 

Maurer uses its graphs to find the face in an image frame. See column 5 lines 47- 
51. In no instance does Maurer calculate 3-D locations of the salient features found in 
the 2-D image. Instead, Maurer has a 2-D graph and a 3-D graph. In both cases, the 
graphs are manipulated, distorted or scaled until they correspond to the image. However, 
distorting a three dimensional graph to conform it to an image is not a calculation of 3-D 
locations of the salient features of the object. As pointed out above, the salient features 
of the object are marked with markers. These markers are placed on the object and are 
not as in Maurer, selected from what is naturally available on the object. By using 
markers and placing the salient features on the face, the invention allows for faster and 
more reliable motion tracking under adverse conditions such as variations in illumination, 
poor video quality and partial inclusion of the features. See applicant's specification, 
page 4, lines 10-21. 

Claim 12 is patentable over the art of record. Claim 12 is amended to define its 
steps as locking onto the markers and detecting loss of lock to relock on the markers. 
Since a portion of the reference relied upon by the Examiner does not show markers, 
claim 12 is patentable over the art of record. 
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Maurer reference fails to show or suggest determining a surface normal for each 
salient feature. As such, claim 13 is patentable over the art of record. The rejection at 
the bottom of page 3 fails to identify any portion of Maurer that shows or suggests 
determining surface normals for salient features. Column 12, lines 3-6 refer only to 
markers in general and do not identify any surface normals. Lines 24-27 in column 12 
only generally refers to facial sensing and fails to show or suggest surface normals. 

The step of determining the surface normals is disclosed in the specification 
beginning at page 1 8, line 5. There the surface normal is defined as a vector 
perpendicular to the surface of the face at the location of the marker. Surface markers are 
an important feature of the invention because they are used to determine the visibility of 
the markers in a 2-D image. The visibility of a marker is determined by the position of 
the marker and the orientation of the face. For example, the surface normal on the edge 
of a facial image becomes less visible as the face turns in the direction of the marker and 
thereby orients the marker away from the camera. By using the surface normals, the 
visibility of markers can be calculated and thus the motion of the original face is more 
accurately tracked. 

Claim 14 is patentable over the art of record for the same reasons given above in 
connection with claim 12. 

Claim 15 is cancelled. 

Claim 16 is patentable over the art of record. It calls for fixing markers to the 
face and tracking the motion of the markers. Claim 16 is rejected based upon Maurer at 
column 3, 12-29. However, that section of Maurer fails to show or suggest fixing 
markers to a face and then tracking the markers. Instead that section of Maurer deals 
with an abstract graph with nodes that may be deformed. The abstract graph and nodes 
are scanned over an image of the face in order to acquire or locate a person's face within 
an image. Thus, not only is the structure of the invention different from Maurer, but the 
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purpose of the markers and the elements are dramatically different from the purpose, 
function and use of the graph in Maurer. 

Claim 17 distinguishes between two kinds of markers, one for global motion and 
another for local motion on the face. Maurer does not distinguish between markers for 
identifying global motion and those for local motion. The portion of Maurer relied upon 
for the rejection discloses only the nodes of the graph. As discussed above, the nodes of 
the graph are not markers and are not placed on the object. 

Claim 1 8 is patentable over the art of record. Maurer fails to show or suggest 
markers that comprise two colors. For similar reasons, claims 19, 20, 21 and 22 are 
patentable over the art of record because Maurer fails to show or suggest contrasting 
colors where the colors are black and white fails to show two concentric circles of 
different colors and fails to show an outer circle with a diameter at least twice the 
diameter of the inner circle. 

Applicants point out that the office action fails to make any finding at all with 
regards to claims 20, 21 and 22. Thus, neither those claims nor claims 18 and 19 are 
shown or suggested by Maurer. The rejection appears to equate dynamic texture 
generation to color. While that may or may not be correct, Maurer still fails to show or 
suggest coloring markers that are on an object. Instead, the dynamic texture generation 
referred to in Maurer is the object itself or rather the image of the object itself. 

The office action is inconsistent with regard to the claim 23. On page 4, the 
action states that column 12, lines 3-5 of Maurer disclose a headset. That is incorrect. 
There is no mention whatsoever of a headset at that location in Maurer. On page 9, 
regarding claim 23, the action states that Maurer and Singh are silent about a headset. 
That is correct. The reference of Ferre discloses a headset with markers that are used in 
connection with surgery. However, the Ferre unit is described in connection with 
medical imaging for computerized axial tomography, magnetic resonance imaging and 
position emission tomography. The markers in Ferre are used to orient and align the 
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surface of a person's face to subsurface, anatomical features inside the person's head. 
The markers are not used in a system for tracking motion of the face. Indeed, applicants 
believe the patient's head is normally immobilized since any motion during surgery 
might be harmful to the patient. 

Similar comments apply for claims 25 and 26. 

Claim 27 is patentable over the art of record. In claim 27 the 2-D images of the 
face that are acquired are acquired with markers in neutral states at different orientations. 
An example of applicant's invention is found in figure 7. In our reference in particular 
column 10, lines 31-50, fails to show or suggest this invention. Maurer is not acquiring 
images with markers on the face at two different orientations. Instead, Maurer is using 
different graphs to find one that most closely matches the single image shown in the 
picture. However, the picture or the object does not have any markers. Moreover, 
Maurer is silent about having two views with markers at different orientations. Maurer is 
also silent about distinguishing between neutral states and action states. 

Claim 28 is patentable over the art of record. Maurer fails to show or suggest two 
views that are orthogonal. Figure 6, item 62 is an image graph. It is not a view of a face 
with markers. The portion of the Maurer specification referred to at column 4, line 61- 
67 is not "orthogonal" or any equivalent. That section of the Maurer specification fails to 
support the rejection. 

Claim 29 is patentable over the art of record. Acquired 2-D images listed in claim 
29 are not particularly shown or suggested by Maurer. In addition, the portions relied 
upon in the rejection fail to support that rejection. Column 3, line 54 through column 4 
line 8 refers to a graph and not to 2-D images such as those specified in the claims. 
While it may be true that Maurer uses a graph to locate eyes, ears, noses and other 
features of a person's face, locating those features is not the same as capturing an image 
of them. Applicants use the captured and stored image in two dimensions to provide 
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three dimensional measurements and generate the surface normals. Maurer fails to show 
or suggest requiring the requisite images specified in claim 29. 

Claim 30 is patentable over the art of record. The portion of Maurer relied upon 
in the rejection is column 12, lines 28-34. That portion of Maurer is directed to 
extracting local features form an image using Maurer' s elastic bunch graph technique. 
However, the particular uses identified in claim 30 are not shown or suggest by that 
portion of Maurer. Figure 14 of Maurer shows only two views, a frontal and a profile 
view. Claim 30 includes several other views. In addition, claim 30 requires that there are 
views of the face with markers in the neutral state. As pointed out above, Maurer has no 
such markers. 

Claim 3 1 is patentable over Maurer. The portion of Maurer relied upon by the 
rejection fails to show or suggest the subject matter of claim 3 1 . It does not show 2-D 
images that include a plurality of views of the face with m arkers in at least one action 
state. Maurer fails to show or suggest markers and fails also to show or suggest 
differences between neutral states and action states. 

Claim 32 is patentable over the art of record. The rejection is clearly erroneous. 
Maurer fails to show or suggest action states as defined in claim 32 where those states are 
captured with markers. Maurer does not distinguish between action and neutral states 
and Maurer has no markers. 

Claim 33 is patentable over the art of record. Maurer does not disclose or suggest 
action states that include the views of different orientations where markers are on a face. 
As stated above, Maurer has no markers and fails to show or suggest action states and 
separate neutral states. 

Claim 34 is dependent upon claim 33 and is patentable for the same reasons given 

above. 
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Claim 35 is amended to delete the word selecting and insert the work marking. It 
is patentable over the art of record because Maurer fails to show or suggest fixing 
markers to a face and then calculating 3-D locations of the markers that are placed on a 
face. Instead, Maurer forms an abstract graph. Forming an abstract graph does not show 
or suggest placing physical markers on a face. 

Claim 36 is not shown or suggested by the art of record. The office action fails to 
properly examine claim 36. On page 8 of the office action, claim 36 is summarily 
rejected on grounds that it has subject matter similar to claims 1 1, 35 and 38. However, 
that is not correct. Claim 36 has particular limitations that are not found in any of those 
claims. Claim 36 expressly calculates 3-D locations of the global and local markers in 
the neutral and calculates the locations of the local markers in each action state. No 
where does Maurer show or suggest making similar or identical calculations. 

Claim 37 is also patentable over the art of record. Again, claim 37 was 
improperly examined and summarily rejected as having similar limitations to claims 11, 
35 and 38. That is incorrect. Claim 37 is dependent upon claim 36 and has numerous 
details not found in claims 1 1, 35 and 38. For example, it includes calculating 3-D 
locations of the markers to conform their 2-D locations in the 2-D images of the face in 
the neutral states in an orthographic projection model. That limitation is not found in the 
other claims nor is it found in Maurer. As such, claim 37 is patentable over the art of 
record. 

Claim 38 is patentable over Maurer. Claim 38 defines a step of calculating 3-D 
locations of local markers in each action state. Maurer does not show or suggest such a 
step and the portion of Maurer relied upon by the rejection is inadequate to show that 
step. In Maurer, column 6, lines 6, column 14, Maurer is referring to his tracking 
technique. Tracking is used to follow the node position in one image to a different node 
position in a subsequent image. However, following a node from one image to the next is 
not the same as estimating the orientation and position of the face in each of the captured 
2-D images in the action state so that they conform to the 2-D and 3-D locations of the 
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global markers under a general perspective projection model. This step of the applicant's 
invention is performed before tracking. The applicant's invention one has to first 
determine the markers are before they are tracked. In addition, claim 38 specifies the 2-D 
- 3-D relationship of the invention. In other words, the 2-D markers locations of the 
captured image are related to the 3-D perspective projection model. This 2-D to 3-D 
conversion is not shown or suggested anywhere in Maurer. Instead, Maurer simply i 
either a 2-D to 2-D system or a 3-D to 3-D system. 



uses 



Claim 39 is patentable over the art of record. Claim 39 was summarily and 
erroneously rejected on grounds that its subject matter was included in claims 1 1, 35 and 
38. That is incorrect. As stated above, claim 33 defines a feature of the invention that 
translates between the 2-D image and the 3-D global motion. By predicting the location 
of the global salient features in a 2-D image detecting the global salient features in the 
image, the invention is able to estimate the 3-D global motion of the face in the 2-D 
image. This is an important feature that efficiently uses 2-D information to track 3-D 
motion. This feature is not shown or suggested by Maurer and is not identical to the 
limitations found in claims 1 1, 35 and 38. 

Claims 40 and 43-45 were erroneously rejected on grounds that they are identical 
to claims 29 and 38. The rejection is erroneous. First of all, claim 40 calls for 
calculating 2-D locations using the prior position of a 2-D image. No such limitation 
appears in the claims 29 or 38. However, Maurer fails to show or suggest such a 
calculation. 



Claims 43-45 appear to be rejected on grounds they include the same images 
found in claim 29. That is clearly erroneous. Claim 43 has steps of predicting and 
detecting. No such steps are found in claim 29 or in claim 38. 

Claim 44 finds placing local markers on eyebrows and lips and in particular on 
the ends of eyebrows and corners of lips and upper and lower centers of each lip (claim 
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45). Such locations are not found in any of the reference claims nor are they found in 
Maurer. 

Claim 46 is amended to change "action values" to -actions states--. Claim 46 is 
not shown or suggested in the art of record nor is it similar to the limitations found in 
claims 1 1 and 35. Nothing in those claims provides the same limitation as predicting the 
locations of local markers using the position from a previous 2-D image. Claim 35 is 
directed to calculating 3-D locations. Claim 35 is not based upon predicting a location 
based upon a prior location. 

Claim 47 is patentable over the art of record. It too is not shown or suggested nor 
does it have similar limitations to claims 1 1 and 35. Claim 47 has limitations not found 
in any of those claims including determining visibility and designing correlation filters 
for local markers. 

Claim 48 has limitations not found in earlier claims and not shown or suggested 
by the art of record. It includes calculating an action vector that represents the weight of 
facial actions in a 2-D image. None of the art of record shows or suggests such a feature. 
Claim 48 was summarily rejected, erroneously so, because it has limitations to claims 1 1 
and 35. However, it has limitations not found in those claims and the office action fails 
to show or suggest any art of record that would prevent the patentability of claim 48. 

Claim 49 is patentable over the art of record. Claim 49 depends upon calculating 
the difference between 2-D locations of local markers in an image. They are in the 
neutral state, modifying them to conform to the orthographic projection. Thereafter, 3-D 
displacements of local markers are calculated with respect to the neutral face. Amount of 
facial action is calculated to conform to 3-D displacements of the local markers. None of 
those steps are shown or suggested by the art of record. In column 6, line 66 through 
column 7, line 14 Maurer describes tracking a node on Gaussian image pyramid. 
However, tracking a node of a Gaussian image that is part of the graph of figure 6 is in no 
way identical to or suggestive of calculating the difference of 2-D locations of local 
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markers. As explained above, Maurer fails to show or suggest placing markers on a 
person's face. Thus, it cannot calculate the difference between markers that it has never 
placed. 

Claim 50 is patentable over the art of record. Any of the limitations in claim 50 
are directed to calculations based upon locations of the local markers. As explained 
above, Maurer has no local markers and the portions of Maurer cited do not show or 
suggest the local markers defined in claim 50. 

Claim 51 is patentable over the art of record. Claim 51 was erroneously rejected 
on grounds that it has similar limitations to claims 1 1 and 35. While it has some 
similarities, it also has differences. These differences were not addressed by the office 
action. Thus, the action is incomplete and is inadequate to reject this claim. 

Claim 51 has a step of predicting 2-D locations. Neither claim 1 1 nor claim 35 
have a similar limitation. Claim 51 as amended is patentable over the art of record on 
grounds that the limitation of fixing markers to the object is not shown in the art of record 
applied to the claim. As stated above, none of the art of record shows or suggests fixing 
markers to the object and performing the further steps of claim 51. Thus, it is patentable 
over the art of record. 

Claim 52 further defines the action state as the maximum local motions of the 
object. None of the art of record defines an action state and also none define the action 
state as the maximum local motion of an object. As such, claim 52 is patentable over the 
art of record. 

Claim 53 more particularly defines that steps of calculating the 3-D locations 
based upon the 2-D image. This particular aspect of the invention is not shown or 
suggested by any of the art of record for the reasons given above. 

Claim 54 is cancelled as its subject matter is now included claim 61. 
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Claim 55 is patentable on grounds that none of the art of record distinguishes 
between global and local salient features and assigns different markers to each set. 



Claim 56 is patentable over the art of record on grounds that the art as applied to 
the claims fails to show or suggest the particular estimating step set forth in the claim. 

Claim 57 is patentable on grounds the art of record fails to show or suggest 
calculating weight vectors representative of marker action. 

Claim 58 is patentable on the same grounds as claim 51 . 

Claim 59 is patentable on grounds that none of the art of record shows markers 
that comprise concentric circles with contrasting colors. 

Claim 60 is patentable on grounds that none of the art of record shows the step of 
detecting that comprises applied elliptical correlation filters in the neighborhood of 
markers. 

In summary, the claims as presently presented are patentable over the prior art and 
a notice of allowance is respectfully requested. 
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